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Creating a Performance Scorecard 


As part of the Missouri Training and Employment Council’s (MTEC) 
efforts to provide an ongoing assessment of the state of Missouri's 
workforce, the Council deployed a performance scorecard for the state’s 
$1.8 billion workforce investment system in 2004. Through the 
Scorecard, the Council encouraged Missouri's workforce investment 
system to focus on value-added services for its customers (students, 
businesses and job seekers) and thereby produce a better return on the 
tax dollars invested in the system. 


The Council worked with the Corporation for a Skilled Workforce (a 
nationally recognized consulting firm) to develop a set of leading and 
lagging indicators to assess the system’s performance. The MTEC 
Education and Training Committee and Missouri's Workforce Policy 
Academy Team met several times in 2004 to refine the Scorecard and 
ensure that the measures were meaningful. Workforce Investment 
Board Chairs and Administrators have also influenced the development 
of the Scorecard. The MTEC Workforce Investment Board Committee 
reviewed and approved the Performance Scorecard on June 18 and full 
Council approved the final format on June 25, 2004. 


The Missouri Economic Research and Information Center (MERIC) 
began the data collection and analysis process for the 2004 
Performance Scorecard report on July 1. Preliminary results were 
presented to the MTEC WIB Committee on September 10 and to the full 
Council on October 5, 2004. 


Based on feedback from these meetings, MERIC refined the 
assessment model and published the 2004 Workforce Investment 
System Performance Scorecard in January of 2005. This report, the 
2005 Workforce Investment System Performance Scorecard, is an 
update of the 2004 report. The Scorecard includes measures in three 
major performance categories: 1) Missouri Career Centers; 2) 
Missouri's Educational System; and 3) Missouri’s Economy. 





Evaluating System Performance 


It is critical that Missouri's state and local workforce investment boards, 
career centers (One-Stop centers), education entities, and economic 
development entities be able to evaluate collective system performance 
and determine whether their integrated efforts are moving toward their 
vision of excellence. Education outcomes reflect success in developing 
a skilled and educated citizenry; skilled and educated citizens become 
productive employees and contribute to the economic prosperity of the 
state; and economic prosperity leads to a quality of life Missouri can use 
to draw both business and new citizens. One way to address this need 
is to talk about a balanced scorecard of performance measures and 
indicators. 


Why Use A Balanced Scorecard Approach 


A balanced scorecard of indicators provides a variety of measures to 
guide planning and tactical decisions. Consider the dashboard of a car. 
It has many indicators of your car’s ultimate ability to take you where you 
want to go. The outcome measure is whether you reached your 
destination on time, and the indicators on your dashboard tell you what 
your odds are of meeting the outcome. If you look solely at the speed 
but ignore the gas gauge, running out of gas could cause you to be late 
even if you maintain a high speed. If you focus on the RPMs and ignore 
engine-warning lights, you may also jeopardize your ability to attain the 
desired outcome. Having a half tank of gas versus a full tank of gas 
doesn’t necessarily mean you are going to fail. You merely use that 
indicator to decide whether you need to buy more gasoline, considering 
how far away you are from your destination. 


Similarly, businesses and governmental agencies must determine what 
measures and indicators they need to examine on a regular basis. The 
balanced scorecard is used, not only by management to observe 
whether the system is on course, but also as a means of 
communication. The indicators clarify policy direction and priorities and 
allow employees to direct their energy toward desired results. 





& Poverty Rate 

© Wage Gain 

© Median Household Income 

© Industry Diversity 

© Value of Goods and Services Exported 
© Gross State Product 

© New Businesses Established 

©) Businesses Declaring Bankruptcy 
©5 Permanent Job Loss 

© New Job Creation 

© Total Labor Force Participation 

©) Unemployment Rate 

©) Non-Violent Crime 


Missouri's Education System Scorecard 


Leading Indicators 
© MAP Scores in K-12 (Communication Arts & Math) 


© Students Enrolled in AP Classes 
©) ACT Scores (Subscale Scores) 
©) TIMSS Scores (Math & Science) 


Lagging Indicators 
© Graduation Rates (Secondary) 
©) Drop Out Rates (Secondary) 
©) Students Entering Post-Secondary Education 
© Students Completing Post-Secondary Education 
&) Remediation Rates (Reading & Math) 
© Adult Literacy Rates 
© Education Attainment 
© GED Attainment 





Outcomes for Employer Customers 
©& Employer Satisfaction 
© Market Penetration 
© Cycle-time 


Outcomes for Individual Customers 
© Customer Satisfaction 
©) Increased Earnings 
© Entered Employment Rate 
© Market Penetration 
© Training-related Credential 
© Employment Retention 








Overview ..... 2-3 
Summary Scorecards ..... 4-5 


Missouri’s Economic System Scorecard..... 8 


Poverty Rate..... 9 

Wage Gain..... 10 

Median Household Income .... . 11 
Industry Diversity ..... 12 

Value of Goods and Services Exported..... 13 
Gross State Product ..... 14 

New Businesses Established .... . 15 
Businesses Declaring Bankruptcy ..... 16 
Permanent Job Loss..... 17 

New Job Creation..... 18 

Total Labor Force Participation... . . 19 
Unemployment Rate..... 20 

Non-Violent Crime..... 21 


Missouri's Education System Scorecard .... . 22 


Leading Indicators..... 23 

MAP Scores in K-12 (Communication Arts & Math)..... 23 
Students Enrolled in AP Classes ..... 24 

ACT Scores (Subscale Scores)..... 25 

TIMSS Scores (Math & Science)..... 26 


Lagging Indicators..... 27 

Graduation Rates (Secondary) ..... 27 

Drop Out Rates (Secondary) ..... 28 

Students Entering Post-Secondary Education..... 29 
Students Completing Post-Secondary Education..... 30 
Remediation Rates (Reading & Math)..... 31 

Adult Literacy Rates..... 32 

Education Attainment..... 33 

GED Attainment ..... 34 





IV. Missouri’s Career Center Scorecard..... 35 


V. 
Vi. 


Outcomes for Employer Customers..... 36 
Employer Satisfaction..... 36 

Market Penetration..... 37 

Cycle-Time..... 38 


Outcomes for Individual Customers ..... 39 
Customer Satisfaction ..... 39 

Increased Earnings..... 40 

Entered Employment Rate..... 41 

Market Penetration..... 42 

Training Related Credential... . . 43 

Employment Retention ..... 44 


Implications & Summary ..... 45-48 


Sources..... 49-52 









The Economic System Scorecard measures Missouri's performance 
against the nation’s on various economic measures. Measures such as 
industry diversity, gross state product, and new businesses established 
reflect the state’s ability to develop, maintain, and support industry 
growth. Other measures such as poverty rate, wage gain, 
unemployment, and median income describe the quality of life that the 
state’s population maintains year after year. 


The data show that Missouri has an extremely diverse mix of industries 
when compared to the most diverse of economies, the national 
economy. Despite this array of industries, 
the state’s economy has felt the impact of 
| the national recession in many ways. 
Median household income has significantly 
declined from the high levels that were seen 
before the tech stock bubble burst in 2000- 
2001. The rate of individuals that are 
unemployed and in poverty has also 
continued to increase since 2000, which is 
consistent with national trends. The number 
of new businesses established in Missouri 
— continues to grow but at a much slower rate 
than seen in previous years. 


Good signs of economic recovery are also shown with the data. The 
value of goods and services produced in the state (GSP) has grown ata 
rate matching the national economy at a time when over half of the other 
states have seen a decline in GSP. The value of goods and services 
exported from Missouri has grown to total $8.9 billion and has increased 
at a rate that exceeds national export growth. The number of non-violent 
crimes reported in the state has decreased somewhat over the past few 
years. A steady gain in the real wages that Missouri workers earn over 
time is also evident in the data. 





The Missouri poverty rate is measured by the percentage of adults 
considered to be in poverty. The standards for poverty are set by the 
Federal government. Poverty threshold is first determined by size of 
family, age of family members, as well as various sources of income 
available to the family. A family whose income is at or above a 
determined poverty threshold is not in poverty. A family whose income is 
below a poverty threshold is determined to be in poverty. 


e Inthe United States, the poverty rate increased 3.31% from the year 
2002 to 2003. From 2003 to 2004 the rate increased 1.60%, a slightly 
smaller increase than shown in the previous years. 


e In Missouri, the poverty rate increased 8.08% from the year 2002 to 
2003. From 2003 to 2004 the rate increased 14.02%, a significantly 
larger increase than shown in the previous years and also 
significantly larger than the national poverty rate increase. 
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The wage gain for workers in Missouri is an average of the total working 
population’s wages for a year. The information is collected annually and 
is converted into real wages to adjust for inflation over time. 


e Inthe United States, the average wage decreased from $38,749 in 
2000 to $38,632 in 2001. From 2001 to 2002, average wages 
decreased slightly from $38,632 to $38,603. In 2004, average 
national wages eventually increased to $39,354. 


e In Missouri, the average wage increased from $34,429 in 2000 to 
$34,581 in 2001. From 2001 to 2002, average Missouri wages 


increased from $34,581 to $34,775. In 2004, average wages slightly 
increased to $34,845. 


Average Annual Wages (adjusted for inflation) 
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The income for a total household in Missouri is averaged to determine 
median household income. The increase or decrease in median 
household income over a period of years for a state can be compared to 
national income trends. This comparison can shed light on the level of 
good paying jobs that are available in an individual state versus the 
entire nation. 


In the United States, median household income increased .57% from 
2000 ($41,990/year) to 2001 ($42,228/year). Median household 
income at the national level also increased by 2.47% from 2003 
($43,318/year) to 2004 ($44,389/year), a faster increase than in 
previous years. 


In Missouri, after sharply decreasing by 8.33% from 2000 
($45,097/year) to 2001 ($41 ,339/year), median household income did 
increase substantially from 2001 to 2003. However, from 2003 to 
2004, median household income in the state sharply fell at a rate of - 
3.81%, a significant deviation from the national trend. 


Missouri Median Household Income 
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The percentage of workers in each major industry sector in a state is a 
reflection of how diverse that state’s workforce is. Each state’s industry 
diversity is compared to the most diverse workforce, the national 
workforce. 


Industry diversity is measured by the Industry Diversification Index (IDI), 
a simple calculation that measures the distribution of various industries 
within a particular area — measured by the proportionate distribution of 
the labor force. The IDI ranges from 0.0 to 1.0, and measures the 
distribution of employment across all major industries. Scores of 0.0 
indicate that the state’s workforce is completely diversified — that the 
workforce is as diverse as the national workforce. Scores of 1.0 indicate 
that the workforce is fully dependent on one type of industry — almost all 
employees are in one industry. 


e Missouri's Industry Diversity as measured on the IDI was -.00001 for 
all industries in 2003 and -.00004 in 2004. 


e These scores show that Missouri's workforce is extremely diverse 
when compared to the national workforce. 


Percentage of Industry vs. Total Workforce 
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The amount of goods and services that an individual state exports can 
be compared to the total amount of exports that are produced by the 
nation as a whole. Dollar amounts are adjusted for inflation over time, 
which reflect how past dollar amounts compare to present day numbers. 


e Between 2000 and 2001, Missouri exports decreased at a rate of 
4.99%, from $6.5 billion to $6.2 billion. These numbers reflect the 
overall drop in the national economy in that national exports also 
decreased at a rate of 6.33% from $780 billion to $731 billion. 

e Both Missouri and national exports increased from 2002 to 2003 
with Missouri numbers rising at a rate of 6.53% from $6.8 billion to 
$7.2 billion. National exports rose from $693 billion to $723 billion. 

e Between 2003 and 2004, Missouri exports rapidly grew at a rate of 
23.61% from $7.2 billion to $8.9 billion. This rebound in exports was 
also felt at the national level with a 12.98% increase, from $724 
billion to $818 billion during the same year. 
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The total amount of goods and services that all Missouri industries 
produce is measured in dollar amounts. A higher dollar amount denotes 
a greater value, which directly reflects that state’s ability to effectively 
produce a range of goods and services. 


e From 2000 to 2001, Missouri’s GSP increased at a rate of .40% from 
$166 billion to $167 billion, which matched the national average for 
growth. From 2003 to 2004, the state’s GSP increased from $181 
billion to $189 billion at a rate of 2.41%, more than keeping up with 
national trends. 
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The growth of new businesses in Missouri is calculated by taking the 
number of new businesses in a given year and dividing it by the total 
number of businesses in existence within that year. 


e From 2000 to 2001, Missouri's new business growth grew 10.15% 
from 130,269 to 130,951 at a rate of 10.15%. The national growth 
rate during the same time period grew 11.55%. 


e Between 2001 and 2002 the number of new businesses in Missouri 
continued to grow from 13,295 to 16,525 at a rate of 12.65%. During 
that time period Missouri outpaced national growth which was 
12.41% 
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The number of businesses that declare bankruptcy in a particular area is 
calculated by subtracting the number of business filings from the total 
number of bankruptcy filings in that area. 


From 2000 to 2001 the number of businesses filing for bankruptcy in 
Missouri grew from 369 to 505 at a rate of 36.86%. Since 2001, this 
number has steadily decreased to only 354 business bankruptcy 
filings in 2004. 


Bankruptcies as a %of Total US Bankruptcies 
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The number of jobs lost in a particular area is defined as the number of 
jobs lost by businesses that close or downsize employment. The 
percentage of the total workforce from lost jobs is measured by dividing 
the number of lost jobs from the total workforce. 


e In Missouri, the number of jobs lost in 2002 steadily increased in all 
four quarters compared to 2001. The number of jobs lost as a 
percentage of the total workforce also steadily increased during all 
four quarters of 2002. 

e In 2003, the number of jobs lost in Missouri in the first and fourth 
quarters (101,109 & 128,857) was down compared to the first and 
fourth quarters of 2002 (107,939 & 145,289). Jobs lost as a 
percentage of the total workforce in 2003 was also down from 2002. 
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The number of jobs created in an area is defined as the number of new 
jobs created by either new businesses or the expansion of employment 
by existing firms. The percentage of the total workforce employed from 
newly created jobs is measured by dividing the number of created jobs 
from the total workforce. 


e In Missouri, the number of jobs created in 2002 steadily increased 
through the first and second quarter and then started to drop off in the 


third and fourth quarters. 
e In 2003, the number of jobs created increased from 


124,159 to 


161,462 jobs from the first to the second quarter. The number 
decreased from the second to the third quarter to 107,238 and 
remained at a lower level throughout the rest of the year. 
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The percentage of the workforce population that participates is defined 
by calculating the civilian labor force as a percentage of the total labor 
force. State total labor force participation rates can be compared to 
national rates to determine whether or not an adequate number of 
civilian workers are being utilized in the workforce. 


e In Missouri, the rate of total labor force participation slightly 
increased .57% from 2000 to 2001, higher than the national rate of 
—.45%. From 2001 to 2002, the rate of Missouri employment 
participation sharply decreased —2.26% which deviated from the 
national rate of -.30%. 

e From 2002 to 2003 the total labor force participation rate in Missouri 
increased .43% which was significantly higher than the decreasing 
national rate of -.60%. However, from 2003 to 2004, the Missouri 
participation rate fell by -1.73%, faster than the national decrease of 
-.30%. 
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The unemployed population is defined as those who are over the age of 
16 who were not working and were available for work. The 
unemployment rate is measured by calculating the number of 
unemployed persons as a percentage of the total labor force. 


e The number of unemployed persons in Missouri has steadily risen 
from 2000 (96,240) to 2004 (172,208). Up to 2003, Missouri 
numbers reflected steadily increasing national rates. 

e From 2003 to 2004, the number of unemployed persons decreased 
nationally. However, from 2003 to 2004 the number of unemployed 
workers in Missouri rose from 168,054 to 172,208. 


Unemployment Rate 


8.00% 


7.00% 





6.00% 








5.00% 





—e— MO 
—s—IL 

—s— KS 
3.00% —e— US 





4.00% 





2.00% 





1.00% 





0.00% 
2000 2001 2002 2003 2004 


20 





Crimes that are non-violent are typically categorized as property crimes. 
Property crimes include: burglary, larceny-theft, and motor vehicle theft. 
The percentage of crime categorized as non-violent is calculated as a 
percentage of total crime reported. 


e In Missouri, the percentage of crimes categorized as non-violent rose 
.11%, from 89.07% in 1999 to 89.18% in 2000. Between 2000 and 
2001, the percentage of non-violent crime in Missouri decreased 
-.57%. 

e Between 2002 and 2003, non-violent crimes as a percentage of total 
crimes rose from 88.30% to 89.46% at a rate of 1.32%. From 2003 to 
2004, non-violent crimes decreased to 2002 levels (88.84%). 


Non-Violent Crime as a % of Total Crime 
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The education scorecard measures Missouri's performance on various 
indicators. Measures such as Missouri Assessment Program (MAP) 
scores, students enrolling in Advanced Placement courses, and ACT 
scores reflect how well Missouri schools prepare students for college as 
well as how they perform in relation to the nation. Other measures such 
as General Educational Development (GED) attainment, adult literacy, 
and educational attainment describe the general educational level of 
Missourians. 


The data show that the state’s education system is showing 
improvement in many areas. Enrollment of high school students in 
Advanced Placement (AP) and dual credit courses has steadily risen 
from 2000 to 2005. The percentage of high school students that 
graduate has also increased over the past four years with the number of 
students dropping out decreasing during the same time period. Other 
areas of strong improvement in the education system include the 
number of students in Missouri that complete their post-secondary 
education. This percentage of post-secondary completion has increased 
from 1999 (49.8%) to 2003 (53.3%). 


Despite the improvement in our schools, the data does indicate that 
certain areas in education still need our attention. Missouri student 
scores on the MAP have risen from 2003 to 2005 for two grade levels. 
However, the percentage of students that have reached the mandated 
level of proficiency for their age group is still less than desirable. Another 
area of concern pertains to the percentage of 
first-time entering freshman in Missouri public 
institutions that are enrolled in remedial math 
and reading classes. The percentage of 
students enrolled in remedial math classes has 
increased from 19% in 1999 to 31% in 2004. 
Likewise, the percentage of students enrolled 
in remedial reading classes has sharply 
increased from 5.9% in 1999 to 11.0% in 2004. 
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Missouri uses the Missouri Assessment Program (MAP) as required by 
the Outstanding Schools Act of 1993 to measure student progress 
toward meeting the Show-Me Standards, 73 rigorous academic 
standards. A five-step scale of “achievement levels” is used to classify 
student performance on the MAP. The level at which a student has 
reached the desired achievement of knowledge and skills called for by 
the Show-Me Standards is called Proficient. The MAP consists of four 
subjects areas: math, communication arts, science, and social studies. 
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Advanced Placement courses are designed to help high school students 
get an early start on their college education and to prepare them for the 
change from high school to college courses. Exams are given at the 
completion of an AP course that, if scored at a 3 or above, is typically 
accepted by colleges and universities for course credit. The numbers of 
students enrolled in both AP and dual credit courses can reflect the 
state’s ability to provide access to AP and dual credit courses. 


Percentage of 11th and 12th Grade Students Enrolled in AP 
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Percentage of Students Enrolled in Dual Credit Courses 


Year Number of Students Number of 11th Percentage of 11th and 12th Grade 
Enrolled in Dual Credit* and 12th Graders Students Enrolled in Dual Credit 
26,784 130,890 20.46% 


27,759 131,407 21.12% 

27,662 135,947 20.35% 

31,869 138,585 23.00% 

35,083 145,730 24.07% 
* These numbers could be inflated due to students enrolling in more than one dual credit class. 
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The ACT test is a widely accepted college entrance exam that assesses 
high school students' general educational development and their ability 

to complete college-level work. The test consists of four multiple-choice 
tests: English, mathematics, reading, and science. 


Percentage of Missouri Students Taking the ACT 
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The Third International Mathematics and Science Study (TIMSS) is an 
extensive international investigation of mathematics and science 
education. The study is sponsored by the International Association for 
the Evaluation of Educational Achievement (IEA) and funded in the U.S. 
by the National Science Foundation (NSF) and the National Center for 
Education Statistics (NCES). Approximately 50 countries participated in 
this survey of education in mathematics and science. 


Comparison of 8th Grade Math Achievement (1999 TIMSS) 
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Comparison of 8th Grade Science Achievement (1999 TIMSS) 
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* Only 23 of 31 countries that participated in TIMSS and TIMSS-R at the 8th grade level met sampling guidelines in both 1995 and 1999. 
Source: TIMSS International Study Center; Mathematics Benchmarking Report, Science Benchmarking Report (TIMSS, 1999). 
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The percentage of high school students that graduate is calculated by 
taking the total number of graduates and dividing it by the number of 
cohort dropouts and graduates. 


e The Missouri graduation rate has steadily risen from 80.1% in 2000 
to the significantly higher rate of 85.7% in 2005. 

e The number of Missouri graduates has likewise increased from the 
2000 total of 52,852 to 57,495 in 2005. 
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The number of students that drop out of high school is measured by 
taking the number of students that drop out divided by beginning of the 
year enrollment (which takes into account transfers) then dividing by 
two. 


e In 2000 the drop out rate for Missouri secondary schools was 4.5% 
(11,716 students). This number has decreased over the years to a 
rate of 3.8% in 2005 (10,341 students). 
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Missouri Drop Out Rates for Black 
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The number of first-time freshmen from Missouri that attend Missouri 
institutions indicates whether a state is an importer or exporter of 
college-going students. States that bring in more students than they 
export may benefit if out-of-state students stay after graduation. 
Conversely, if a state exports large numbers of students to other states, 
they may lose talented young residents if they do not return after 
graduation. These data include all first-time freshmen — not just those 
attending directly out of high school. 


Students Completing Post-Secondary Education 
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The percentage of first-time full-time bachelor’s degree-seeking students 
earning a bachelor’s degree within six years is a measure of the 
efficiency with which students complete college. It is also a good 
measure of how well students persist toward gaining a degree. A high 
value on this measure benefits a state in two important ways: 1) it leads 
to higher degree production and a better educated population, and 2) 
the postsecondary system is functioning better or students are moving 
through the system at higher rates allowing more room for others to 
enter. This statistic does not account for transfers across institutions. 


e From 1999 to 2003 the percentage of Missouri students completing 


post-secondary education steadily rose from 49.8% in 1999 to 53.3% 
in 2003. 
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The percentage of students that require remedial classes as they enter 
public post- secondary institutions is calculated by taking the number of 
first-time entering freshman enrolled in remedial math and reading 
classes and dividing it by the total number of first-time entering freshman 
in public schools. 


e The remediation rates for Missouri first-time freshman enrolled in 
remedial reading classes stayed relatively stable from 1999 to 2003, 
but sharply increased from 5.9% to 11.0% in 2004. 


e Missouri has seen a steady increase in the percentage of first-time 
freshman enrolled in remedial math courses, from 19% in 1999 to 
31% in 2004. 


Percentage of First-time Freshman Enrolled in Remedial 
Classes at Missouri Public Institutions 
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Literacy consists of a variety of skills that develop over the course of an 
individual's lifetime. Adults that score lowest on three literacy scales 
(prose, document, and quantitative literacy) are labeled as having Below 
Basic Literacy. Many adults in this category can perform simple tasks 
involving uncomplicated text and documents. Other adults are unable to 
complete these types of tasks while others have such limited skills that 
they cannot respond to a survey. 


e In 2003, the percentage of adults in Missouri with Below Basic Prose 
Literacy was 7%, significantly lower than the national rate of 14%. For 
adults with Below Basic Document Literacy, Missouri had a rate of 
8%, again, significantly lower than the national rate of 12%. The 
percentage of Missouri adults with Below Basic Quantitative Literacy 
(15%) was also significantly lower than national levels (21%). 
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The percentage of adults over the age of 25 with a post-secondary 
degree is calculated by taking the number of individuals over the age of 
25 with a post-secondary degree and dividing it by the overall population 
over the age of 25. 


e The percentage of Missourians over the age of 25 with a post- 
secondary degree has risen from 2000 (28.2%) to 2004 (30.3%). This 
increase in post-secondary degrees mirrors national trends during the 
same period. 

e However, Missouri lagged behind national and surrounding state 
levels of post-secondary degree attainment between 2000 and 2004. 
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The General Educational Development (GED) test consists of five 
multiple-choice tests that cover reading, mathematics, social studies, 
science, and writing skills. Most post-secondary institutions and 
employers accept a GED certificate as being equivalent to a high school 
diploma. Those who are at least 16 years old and are no longer enrolled 
in high school are eligible to take the GED test. 


e The number of students in Missouri that take the GED has decreased 
from 1999 (13,821 students) to 2003 (10,834 students). This mirrors 
declining numbers of GED test takers in surrounding states. 


e The percentage of Missouri students that take the GED and pass has 
risen from 74.7% in 1999 to 76.4% in 2003. 


Percentage of Students Taking GED and Passing 
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The Career Center System Scorecard measures Missouri's Career Center 
System performance on various Career Center performance measures. 
Measures such as employer/customer satisfaction, cycle time, and market 
penetration reflect the career center system’s ability to 
serve and support the needs of both employer and 
individual customers. Other measures such as cycle 
time and employment retention describe the career 
center system’s ability to meet the skill and labor 
demands of employers and individual customers ina 
timely manner. 


The data show that the Missouri Career Center 
System’s performance is comparable to and, in many 
cases, better than average national performance on 

' various measures. The employment rates of those 
adults, dislocated workers and older youth (aged 19-21 
years) that entered employment by the end of the first 
quarter after exiting the Missouri Career Center System improved from 
2003 to 2004 and were above the national averages. The percentage of 
adults, dislocated workers, and older youth that maintained employment up 
to the third quarter after exiting was comparable to national percentages in 
2004. Likewise, the percentage of younger youth in Missouri that were 
found in post-secondary education, advanced training, employment, 
military service, or qualified apprenticeships in the third quarter after exit in 
2004 was comparable to national levels. 


However, the results also point toward areas of concern. The percentage 
of all employers that have job openings served through the Missouri’s 
Great Hires registration system in a given year has slightly decreased from 
2003 to 2005. Likewise, the percentage of all job seekers that acquire new 
jobs through the Missouri's Great Hires registration system in a given year 
has slightly decreased from 2003 to 2005. In addition, both employer and 
individual satisfaction with the Career Center System declined from 2003 to 
2004. 
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To meet the customer satisfaction requirements of the Worforce 
Investment Act, the Workforce Investment System uses customer 
satisfaction surveys. The surveys are given to a sample of different 
exiter groups in each business quarter. Employer satisfaction is 
measured by taking the weighted average of employer ratings on each 
of three questions regarding overall satisfaction, reported on a 0-100 
point scale. The score is a weighted average, not a percentage. 


e In Missouri, employer customer satisfaction has decreased from 
74 in 2000 to 69.6 in 2004, with a large decline in 2004 from 2003. 

e The U.S. employer customer satisfaction rate (72.5) was higher 
than Missouri's rate (69.6) in 2004. 


Employer Satisfaction with Career Center System 
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The employer market penetration rate is the percentage of all employers 
with job openings served through the Missouri's Great Hires registration 
system in a given year. 


e In Missouri, the employer market penetration rate has slightly 
decreased from 27.4% in June of 2003 to 27.2% in June of 2005. 
The employer market penetration rate averaged 29.6% from July 
2004 to June 2005. 
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Cycle-time is measured by totaling the number of days between the time 
that a job is listed in the Workforce Investment System and the time that 
the job is filled through the System. 


e Missouri's Workforce Investment System’s cycle-time is an average 
of 21 days for the entire state. 

e Different Workforce Investment Act regions have cycle-times ranging 
from 15 days in the Southwest Region to 24 days in both the Kansas 
City and Vicinity and Ozark Regions. 
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To meet the customer satisfaction requirements of the Worforce 
Investment Act, the Workforce Investment System uses customer 
satisfaction surveys. The surveys are given to a sample of different 
exiter groups in each business quarter. Individual participant satisfaction 
is measured by taking the weighted average of participant ratings on 
each of three questions regarding overall satisfaction which are reported 
on a 0-100 point scale. The score is a weighted average, nota 
percentage. 


e In Missouri, individual customer satisfaction has decreased from an 
average score of 71 in 2000 to 66.5 in 2004. 

e In 2004 the U.S. had slightly higher levels of customer satisfaction 
(76.8) than Missouri (66.5). 
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The average earnings change Is measured by taking an individual’s total 
post-program earnings (earnings in Quarter 2 + Quarter 3 after exit) 
minus pre-program earnings (earnings in Quarter 2 + Quarter 3 prior to 
registration) divided by the number of adults who exit during the quarter. 
The adults measure only includes adults employed in the first quarter 
after exiting the system. The dislocated workers measure Is a rate that 
includes dislocated workers, employed in the first quarter after exit, 
using the individual’s last day of employment instead of date of 
registration. The older youth measure includes those who are employed 
in the first quarter after exit and who are not enrolled in post-secondary 
education or advanced training in the third quarter after exit. 


e Earnings change in six months 
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The entered employment rate is calculated by taking the number of 
individuals that have entered employment by the end of the first quarter 
after exit divided by the number of individuals that exit during the first 
quarter. 


Adults who are employed at the time of registration are excluded from 
this measure. All dislocated workers are included in this measure. Older 
youth who are employed at registration or not employed but are in only 
post-secondary education or advanced training in the first quarter after 
exit are also excluded from this measure. 


e In Missouri, the entered 
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The individual customer market penetration rate is the percentage of all 
job seekers that acquire new jobs that are served through the Missouri's 
Great Hires registration system in a given year. 


e In Missouri, the customer market penetration rate has been trending 
slightly downward. The customer market penetration rate decreased 
slightly from 52.9% in June of 2003 to 51.7% in June of 2005. 
Overall, the rate averaged 57.5% from July 2004 to June 2005. 
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The employment and credential rate for adults and dislocated workers is 
calculated by taking the number of individuals that were employed in the 
first quarter after exit and received a credential by the end of the third 
quarter after exit divided by the number of individuals who exited 
services during the quarter. Only adults and dislocated workers who 
received training services are included. The credential rate for older 
youth is calculated by taking the number of older youth in employment, 
post-secondary education, or advanced training in the first quarter after 
exit and received a credential by the end of the third quarter after exit 
divided by the number of older youth who exit during the quarter. The 
diploma or equivalent rate for younger youth is calculated by taking the 
number of younger youth who attained secondary school diploma or 
equivalent by the end of the first quarter after exit divided by the number 
of younger youth who exit during the quarter (except those still in 
secondary school at exit). 


Employment & Credential Rate Employment & Credential Rate 
(Adults) (Dislocated Workers) 


160% 
ME Actual Rate 100 150% fmm Actual Rate 


110% 80 


[Negotiated 60 100% [—Negotiated 
60% Rate Rate 


40 
10% -+—%of 50% -«—-%of 


Negotiated 20 Negotiated 
-40° Rate Rate 
40% 0 0% 
2000 2001 2002 2003 2004 





2000 2001 2002 2003 2004 


Credential Rate Diploma or Equivalent Rate 


(Older Youth - age 19-21) (Younger Youth - age 14-18) 


160% mm Actual 160% 
140% Rate GS Actual 


120% 110% Rate 
100% (~~ 1 Negotiate 


80% d Rate 60% [1 Negotiated 
Rate 

60% -a«-%of : 

40% Negotiate 10% s— % of 

20% d Rate Negotiated 

0% 40% Rate 


2000 2001 2002 2003 2004 rick aaieamecraeataai 





The employment retention rates for adults and dislocated workers are 
calculated by taking the number of adults or dislocated workers 
employed in the third quarter after exit divided by the number of adults 
or dislocated workers who exit during the quarter. Only those adults and 
dislocated workers employed in the first quarter after exit are included in 
the measure. The employment retention rate for older youth is 
calculated similarly to the adult and dislocated worker rates but includes 
only those older youth who are employed in the first quarter after exit 
and who are not enrolled in post-secondary education or advanced 
training in the third quarter after exit. The younger youth retention rate is 
calculated by taking the number of younger youth found in post- 
secondary education, advanced training, employment, military service, 
or qualified apprenticeships divided by the number of younger youth who 
exit during the quarter. 
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An increasingly knowledge-based and technology-driven economy is 
emerging in the 21* century. The Workforce Investment System in 
Missouri plays a critical role in advancing acquisition of the advanced 
knowledge and skills needed for both individuals and businesses to be 
competitive in this new economy. To that end, the Workforce Investment 
System is in the process of being reformed and revitalized in order to 
meet such demands. 


The results of this Workforce Investment System Performance 
Scorecard report are somewhat mixed in terms of where Missouri’s 
Economic, Education, and Career Center systems are in relation to the 
rest of the nation, and in some cases, the world. The data show that 
Missouri has an extremely diverse mix of industries when compared to 
the most diverse of economies, the national economy. Despite this array 
of industries, the state’s economy has felt the impact of the national 
recession in terms of a declining median household income, increasing 
unemployment and poverty rates since 2000, all of which seem to be 
consistent with national economic trends. Good signs of economic 
recovery are also shown with the data. The value of goods and services 
produced in the state (GSP) has grown at a rate matching the national 
economy at a time when over half of the other states have seen a 
decline in GSP. The number of non-violent crimes reported in the state 
has also decreased over the past few years. 


The report also shows that the state’s education system is showing 
improvement in many areas. Enrollment of high school students in AP 
and dual credit courses has steadily risen from 2000 to 2005. Other 
areas of strong improvement in the education system include the 
number of students in Missouri that complete their post-secondary 
education, which has steadily increased from 1999 to 2003. 

Despite the improvement in our schools, the data does indicate that 
certain areas in education still need our attention. Missouri student 
scores on the MAP have risen from 2003 to 2005 for two grade levels. 
However, the percentage of students that have reached the mandated 
level of proficiency for their age group is still less than desirable. 
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Another area of concern pertains to the percentage of first time entering 
freshman in Missouri public institutions that are enrolled in remedial 
math and reading classes. From 1999 to 2005 the percentage of 
students enrolled in remedial math classes has increased from 19% to 
31%. 


The data show that the Missouri Career Center System’s performance is 
comparable to and, in many cases, better than average national 
performance on various measures. The percentage of adults, dislocated 
workers, and older youth (aged 19-21 years) that maintained 
employment up to the third quarter after exiting the Missouri Career 
Center System was comparable to national percentages in 2004. The 
number of adults, dislocated workers, and younger youth (aged 14-18 
years) that utilize the Missouri Career Centers that go on to receive a 
credential or attain a secondary school diploma or equivalent after 
exiting the system were also comparable to national levels in 2004. 
However, the results also point toward areas that the Career Center 
System should be concerned with. The percentage of all employers with 
job openings served through the Missouri’s Great Hires registration system 
in a given year has slightly decreased from 2003 to 2005. In addition, both 
employer and individual satisfaction with the Career Center System 
declined from 2003 to 2004. 


As noted in the Missouri State of the Workforce Report (2004) Missouri 
has much work to do to build a workforce with the skills required to 
assist their employers to compete in the 21st Century economy. The key 
to Missouri's continued success will be its ability to effectively integrate 
the actions of the business, education and workforce sectors. While 
efforts are underway to work toward common goals, the linkages 
between the three sectors must be accentuated and leveraged for 
success. Missouri is focused on the needs of business and a culture of 
life-long learning (skill refinement) is emerging. Adaptive systems are 
being designed to provide more meaningful information for individuals (a 
broader set of career options) and for businesses (enhanced market and 
workforce data). 
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To that end, the Missouri Training and Employment Council have made 
several recommendations. First, the percentage of citizens who are 
highly literate (reading, comprehension and math skills at the 11th grade 
level or above) must increase significantly. To be competitive in the new 
economy, the workforce must have strong basic skills and have the 
capacity to benefit from training. 


The Missouri Training and Employment Council has identified 
recommendations to expand participation in literacy programs: the State 
must provide strong support for an education continuum of pre-school 
through higher education, literacy instruction should be embedded in all 
adult training programs, a common workforce readiness credentialing 
system for Missouri must be promoted, Missouri Career Centers must 
assist in identifying those in need of literacy training, and support efforts 
by the State’s higher education institutions in developing and promoting 
literacy improvement programs in the communities they serve. 


High school graduation requirements need to be more rigorous and 
should include a nationally recognized work-readiness certification. This 
initiative must be linked with a more proactive policy to strengthen 
teacher preparedness. Adults should also be engaged in continuous 
learning (skills development). Career education and the 
community/technical college system must be expanded and curricula 
targeted to the just-in-time skill standards, certifications, or licensing 
requirements of business and industry. Uniform articulation and dual 
credit mechanisms must be established between and among secondary 
schools, community college, and university levels to provide degree 
credit for skill-based education and training. 


Missouri's workforce development system should strive to increase the 
labor force participation of those persons traditionally underserved by 
Missouri's labor market; specifically persons of low-income, women, ex- 
offenders, at-risk youth, young minority males, and persons with 
disabilities. Missouri must initiate a greater interagency effort to link 
separate programs into a continuum of integrated services, supported by 
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mentoring and individual-based support services, to enable clients to 
participate in skills-based training and/or employment retention 
programs. In addition, community and technical colleges must be highly 
responsive to the short-term and just-in-time training needs of business 
and industry. Such responsiveness will assist both the business and the 
individuals being trained. A public awareness campaign is needed to 
raise Missourians’ aspirations and expectations for education and 
training and their relation to economic survival and growth. States are 
where they are in terms of education, literacy, lifelong learning, and 
economic conditions because of individual and organizational behaviors. 
Those behaviors are shaped by perceptions of what is important and 
has value. The only way behaviors will change is if perceptions are 
changed. 
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Economic Scorecard 


Poverty Rate - U.S. Census Bureau, Current Population Survey 
(Historical Tables), www.census.gov. 


Wage Gain - U.S. Bureau of Labor Statistics, Quarterly Census of 
Employment & Wages, www.bls.gov. 


Median Household Income - U.S. Census Bureau, Current Population 
Survey, WWW.Census.gov. 


Industry Diversity - U.S. Bureau of Labor Statistics, Current Employment 
Statistics, www.bls.gov. 


Value of Goods and Services Exported - World Institute for Strategic 
Economic Research (WISER), www. wisertrade.org. 


Gross State Product — U.S. Bureau of Economic Analysis, 
www.bea.doc.gov. 


New Businesses Established — U.S. Census Bureau, Statistics of U.S. 
Businesses (SUSB), www.census.gov/csd/susb/susb.htm. 


Businesses Declaring Bankruptcy — Administrative Office of U.S. Courts, 
www.uscourts.gov/bnkrpctystats/bankruptcystats.htm. 


Permanent Job Loss - U.S. Census Bureau, Local Employment 
Dynamics, /ehd.dsd.census.gov/led/00. 


New Job Creation - U.S. Census Bureau, Local Employment Dynamics, 
lehd.dsd.census.gov/led/00. 


Total Labor Force Participation - U.S. Census Bureau, Current 
Population Survey, www.census.gov. 
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Unemployment Rate — U.S. Bureau of Labor Statistics, Local Area 
Unemployment Statistics (LAUS), www.bls.gov/lau/. 


Non-Violent Crime — U.S. Federal Bureau of Investigation, Uniform 
Crime Reports, www. fbi.gov. 


Education Scorecard 


MAP Scores in K-12 (Communication Arts & Math) — Missouri Dept. of 
Elementary and Secondary Education (DESE), Annual Report of 
School Data, dese.mo.gov/schooldata/. 


Students Enrolled in AP Classes — The College Board, 
www.collegeboard.com. 


ACT Scores (Subscale Scores) - Missouri Dept. of Elementary and 
Secondary Education, Core Data, dese.mo.gov/schooldata/. 


TIMSS Scores (Math & Science) — Third International Mathematics and 


Science Study, Mathematics Benchmarking Report (TIMSS, 2003), 
Science Benchmarking Report (TIMSS, 2003), ustimss.msu.edu. 


Graduation Rates (Secondary) - Missouri Dept. of Elementary and 
Secondary Education, Core Data, dese.mo.gov/schooldata/. 


Drop Out Rates (Secondary) - Missouri Dept. of Elementary and 
Secondary Education, Core Data, dese.mo.gov/schooldata/. 


Student Entering Post-Secondary Education - Postsecondary Education 


Opportunity Data, referenced at the National Center for Higher 
Education Management Systems website, www. higheredinfo.org. 
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Students Completing Post-Secondary Education — National Center for 
Education Statistics, IPEDS Completion Survey, 
http://nces.ed.gov/ipeds/. 


Remediation Rates (Reading & Math) — Missouri Dept. of Higher 
Education, EMAS Data. 


Adult Literacy Rates - National Center for Education Statistics, National 
Assessment of Adult Literacy Survey, http://nces.ed.gov/. 


Education Attainment — U.S. Census Bureau, Decennial Census Survey, 
WWW.C@NSUS.gOV. 


GED Attainment - GED Annual Statistical Reports, GED Testing 
Service, American Council on Education, www.acenet.edu. 


Career Center System Measures & Indicators 


Outcomes for Employer Customers 
Employer Satisfaction — U.S. Department of Labor, Employment & 


Training Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 


Market Penetration — Missouri Department of Workforce Development 
Planning & Research Group. 


Cycle-Time — Missouri Department of Workforce Development Planning 
& Research Group. 


Outcomes for Individual Customers 

Customer Satisfaction — U.S. Department of Labor, Employment & 
Training Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 
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Increased Earnings — U.S. Department of Labor, Employment & Training 
Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 


Entered Employment Rate — U.S. Department of Labor, Employment & 
Training Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 


Market Penetration — Missouri Department of Workforce Development 
Planning & Research Group. 


Training Related Credential — U.S. Department of Labor, Employment & 
Training Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 


Employment Retention — U.S. Department of Labor, Employment & 


Training Administration, WIA State Annual Reports & Summaries, 
www.doleta.gov. 
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